We study a bandit problem where observations from each arm have anexponential family distribution and different arms are assigned independentconjugate priors. At each of n stages, one arm is to be selected based on pastobservations. The goal is to find a strategy that maximizes the expecteddiscounted sum of the $n$ observations. Two structural results hold in broadgenerality: (i) for a fixed prior weight, an arm becomes more desirable as itsprior mean increases; (ii) for a fixed prior mean, an arm becomes moredesirable as its prior weight decreases. These generalize and unify severalresults in the literature concerning specific problems including Bernoulli andnormal bandits. The second result captures an aspect of theexploration-exploitation dilemma in precise terms: given the same immediatepayoff, the less one knows about an arm, the more desirable it becomes becausethere remains more information to be gained when selecting that arm. ForBernoulli and normal bandits we also obtain extensions to nonconjugate priors.
展开▼
机译：我们研究了一个土匪问题，其中每个分支的观测值都有指数族分布，并且不同分支分配了独立的共轭先验。在n个阶段的每个阶段，将根据过去的观察选择一只手臂。目标是找到一种策略，以最大程度地提高预期的$ n $观测值的折现和。广义上有两个结构性结果：（i）对于固定的先验重量，手臂的平均优先级越高，就越需要手臂； （ii）对于固定的先验平均值，随着先验重量的减少，手臂变得更加理想。这些概括和统一了有关伯努利和正常土匪等特定问题的文献中的一些结果。第二个结果精确地捕捉了开发-开发难题的一个方面：给定相同的即时收益，人们对一条手臂的了解越少，它就越受欢迎，因为在选择该手臂时仍有更多信息要获取。对于伯努利和正常土匪，我们还获得了非共轭先验的扩展。
展开▼
